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TITLE OF THE INVENTION 

VIDEO-INF C)RMAJjm^ AND 
VIDEO-INFORMATION DECODING METHOD 



BACKGROUND OF THE INVENTION 
SUMMARY OF THE INVENTION 

[0001] The present invention, like recently developed standard JVT (ITU-T Rec. H. 264 | 
ISO/IEC 14496-10 AVC), relates to a method, apparatus, and program for encoding video 
information, and a method, apparatus, and program for decoding video information used 
when video information (bitstream) is received through a network medium such as satellite 
broadcasting, a cable TV or the Internet or when video information is processed on a storage 
medium such as an optical disc, a magnetic disk or a flash memory, the video information 
compressed through orthogonal transformation such as discrete cosine transform or the 
Karhunen Loeve transform, and motion compensation. 
DISCUSSION OF THE BACKGROUND 

[0002] Recently, for both of broadcasting stations providing information and homes 
receiving the information, it has become common to use devices that adopt a method such as 
MPEG to compress video information through orthogonal transformation such as discrete 
cosine transform, and motion compensation, utilizing the redundancy of the video 
information, for efficient information transmission and storage, by taking the video 
information as digital information. 

[0003] Especially, the MPEG2 (ISO/IEC 13818-2) is defined as a general video encoding 
method, and is widely used as an application for professionals and for consumers since it can 
treat interlaced images and progressively scanned images, and standard resolution video and 
high resolution video. By using the MPEG2 compression method, a high compression rate 
and high quality of video can be realized, for example, by assigning interlaced images of 
standard resolution of 720 x 480 pixels abit rate of 4 to 8 Mbps, or by assigning 
progressively scanned images of high resolution of 1920 x 1088 pixels a bit rate of 18 to 22 
Mbps. 

[0004] The MPEG2 mainly encodes high quality video for broadcasting and does not cope 
with a bit rate lower than that used by the MPEG1, that is, an encoding method with a high 
compression rate. However, it was expected that popularization of mobile terminals would 
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bring high needs of such an encoding method, and therefore the MPEG4 encoding system 
was standardized. As to a video encoding method, its standard was approved as international 
standard ISO/IEC 14496-2 in December 1998. 

[0005] In addition, recently, with video encoding for video conferencing as a first desired 
usage, a method called JVT (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC) is being 
standardized. Compared with conventional encoding systems such as MPEG2 and MPEG4, 
it is known that the JVT can provide higher encoding efficiency although it requires more 
operations for encoding and decoding. 

[0006] Fig. 8 shows a general construction of a video-information encoding apparatus that 
realizes video compression with orthogonal transformation such as the discrete cosine 
transform or the Karhunen Loeve transform, and motion compensation. As shown in Fig. 8, 
a video-information encoding apparatus 100 is composed of an A/D converter 101, a screen 
rearrangement buffer 102, an adder 103, an orthogonal transformation unit 104, a 
quantization unit 105, a reverse encoding unit 106, a storage buffer 107, a dequantization unit 
108, an inverse orthogonal transformation unit 109, a frame memory 1 10, a motion 
prediction/compensation unit 111, and a rate control unit 112. 

[0007] Referring to Fig. 8, the A/D converter 101 converts an input video signal into a 
digital signal. The screen rearrangement buffer 102 rearranges the frames according to the 
GOP (Group of Pictures) of video compression information given from the A/D converter 
101 . The screen rearrangement buffer 102 gives the orthogonal transformation unit 104 
video information on the entire frames of images to be intra-encoded (within image encoded). 
The orthogonal transformation unit 104 applies an orthogonal transformation, such as the 
discrete cosine transform or the Karhunen Loeve transform, to the video information and 
gives a transform coefficient to the quantization unit 105. The quantization unit 105 
performs a quantization process on the transform coefficient given from the orthogonal 
transformation unit 104. 

[0008] The reverse encoding unit 106 determines an encoding mode based on the quantized 
transform coefficient, which was supplied by the quantization unit 105, and a quantization 
scale, and applies variable-length coding or reverse encoding such as arithmetic coding to the 
encoding mode to thereby create information to be inserted in the header part in each unit of 
encoded video. The encoded encoding mode is given and stored into the storage buffer 107. 
This encoded encoding mode is output as video compression information. 
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[0009] In addition, the reverse encoding unit 106 applies variable-length coding or reverse 
encoding such as arithmetic coding, to the quantized transform coefficient and gives the 
encoded transform coefficient to the storage buffer 107 to store it therein. This encoded 
transform coefficient is output as video compression information. 

[0010] The quantization unit 105 operates under the control of the rate control unit 1 12. The 
quantization unit 105 gives the quantized transform coefficient to the dequantization umt 108, 
which performs dequantization on the transform coefficient. The inverse orthogonal 
transformation unit 109 creates decoding video information by applying an inverse 
orthogonal transformation process to the dequantized transform coefficient, and gives the 
information to the frame memory 1 10 to store it therein. 

[0011] On the other hand, the screen rearrangement buffer 102 gives the motion 
prediction/compensation unit 1 1 1 video information on an image to be inter-encoded 
(between images encoded). The motion prediction/compensation unit 1 1 1 retrieves video 
information used for reference simultaneously from the frame memory 1 10 and performs a 
motion prediction/compensation process to create reference video information. The motion 
prediction/compensation unit 1 1 1 gives the reference video information to the adder 103, 
which then converts the reference video information into a differential signal from the video 
information. At the same time, the motion prediction/compensation unit 1 1 1 gives motion 
vector information to the reverse encoding unit 1 06. 

[0012] The reverse encoding unit 106 determines an encoding mode based on the quantized 
transform coefficient, which was given from the quantization unit 105, the quantization scale, 
the motion vector information given from the motion prediction/compensation unit 1 1 1, etc., 
and applies variable-length coding or reverse encoding such as arithmetic coding to the 
encoding mode, to thereby create information to be inserted into the header in a unit of 
encoded video. The encoded encoding mode is given to the storage buffer 107 to be stored 
therein. The encoded encoding mode is output as video compression information. 
[0013] The reverse encoding unit 106 applies variable-length coding or the reverse encoding 
process such as arithmetic coding to the motion vector information to create information to be 
inserted in the header part in a unit of encoded video. 

[0014] In the inter-encoding, video information to be input into the orthogonal 
transformation unit 104 is a differential signal obtained by the adder 103, which is different 
from the intra-encoding. Since other processes are the same as in the case of the video 
compression information to be intra-encoded, its explanation will be omitted. 
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10015] Next, Fig. 9 shows a general construction of a video-information decoding apparatus 
corresponding to the aforementioned video-information encoding apparatus 100. As shown 
in Fig 9 the video information decoding apparatus 120 is composed of a storage buffer 121, 
a reverse decoding unit 122, a dequantization unit 123, an inverse orthogonal transformation 
unit 124, an adder 125, a screen rearrangement buffer 126, a D/A converter 127, a motion 
prediction/compensation unit 128, and a frame memory 129. 

[0016] The storage buffer 121 temporarily stores input video compression information, and 
then transfers the information to the reverse decoding unit 122. The reverse decoding unit 
122 applies variable-length decoding or a process such as arithmetic decoding to the video 
compression information based on the prescribed format of the video compression 
information, obtains the encoding mode information from its header part, and gives the 
information to the dequantization unit 123. Similarly, the reverse decoding unit 122 obtains 
the quantized transform coefficient and gives it to the dequantization unit 123. In a case in 
which the frame has been subjected to the inter-encoding, the reverse decoding unit 122 
decodes motion vector information stored in the header part of the video compression 
information as well, and gives the information to the motion prediction/compensation unit 
128. 

[0017] The dequantization unit 123 dequantizes the quantized transform coefficient supplied 
from the reverse decoding unit 122, and gives the transform coefficient to the inverse 
orthogonal transformation unit 124. The inverse orthogonal transformation unit 124 applies 
inverse orthogonal transformation, such as inverse discrete cosine transform or inverse 
Karhunen Loeve transform, to the transform coefficient based on the prescribed format of the 
video compression information. 

[0018] In a case in which the frame has been subjected to the intra-encoding, on the other 
hand, the video information subjected to the inverse orthogonal transformation is stored in the 
screen rearrangement buffer 126, and then is output after a D/A conversion process by the 
D/A converter 127. 

[0019] In a case in which the frame has been subjected to the inter-encoding, the motion 
prediction/compensation unit 128 creates a reference image based on the motion vector 
information subjected to the reverse decoding and the video information stored in the frame 
memory 129, and gives the image to the adder 125. The adder 125 adds this reference image 
and the output of the inverse orthogonal transformation unit 124. Since other processes are 
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performed in the same way to the case of the frame subjected to the intra-encoding, its 
explanation will be omitted. 

[0020] Now, the reverse encoding unit 106 under the JVT will be described in detail. As 
shown in Fig. 10, the reverse encoding unit 106 under the JVT adopts one reverse encoding 
out of arithmetic coding called CABAC (Context-based Adaptive Binary Arithmetic Coding) 
and variable-length coding called CAVLC (Context-based Adaptive Variable Length 
Coding), for a symbol such as mode information, motion information, and quantized 
coefficient information, which are input from the quantization unit 105 and the motion 
prediction/compensation unit 1 1 1 , and outputs video compression information (bitstream). 
Based on CABAC/CAVLC selection information in Fig. 10, it is judged which reverse 
encoding is used. This CABAC/CAVLC selection information is determined by the video- 
information encoding apparatus 100 and is output by being embedded in a bitstream as 
header information. 

[0021] First the CABAC system in the reverse encoding unit 106 is shown in Fig. 11. As 
shown in Fig. 1 1 , mode information, motion information, and quantized transform coefficient 
information input from the quantization unit 105 and the motion prediction/compensation 
unit 1 1 1 are input into a binarization unit 131 as multi-valued symbols. The binarization unit 
131 converts the multi-valued symbols into a binary symbol string of an arbitrary length 
under a predetermined rule. This binary symbol string is input into a CABAC encoding unit 
133 and the CABAC encoding unit 133 applies binary symbol arithmetic coding to the input 
binary symbols, and outputs the encoded resultant as a bitstream to the storage buffer 1 07. A 
Context operation unit 132 calculates Context based on the symbol information input to the 
binarization unit 131 and the binary symbols output from the binarization unit 131, and inputs 
the Context to the CABAC encoding unit 133. A Context memory group 135 of the Context 
operation unit 132 stores Context which is updated, as occasion arises, during an encoding 
process, and the initial state of Context to be used for a reset. 

[0022] Next the CAVLC system in the reverse encoding unit 106 is shown in Fig. 12. As 
shown in Fig. 12, mode information, motion information, and quantized transform coefficient 
information input from the quantization unit 105 and the motion prediction/compensation 
unit 1 1 1 are input in a CAVLC encoding unit 140 as multi-valued symbols. Like the 
variable-length coding adopted by the conventional MPEG, the CAVLC encoding unit 1 40 
applies a variable-length coding table to the input multi-valued symbols, and outputs a 
bitstream. A Context storage unit 141 stores information already encoded in the CAVLC 
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encoding unit 140, for example, the number of coefficients of non-zero in blocks already 
processed as well as in blocks being processed, the value of a coefficient encoded 
immediately before this time, and so on. The C AVLC encoding unit 1 40 is able to change a 
variable-length coding table to be applied for symbols, based on information from the 
Context storage unit 141 . It should be noted that the Context storage unit 141 stores the 
initial state of Context to be used for a reset as well. The output bitstream is input into the 
storage buffer 107. 

[0023] Similarly, the reverse decoding unit 122 under the JVT will be described in detail. 
Similarly to the reverse encoding unit 106, the reverse decoding unit 122 under the JVT 
applies one reverse decoding out of CABAC and CAVLC to an input bitstream, as shown m 
Fig 13. By reading the CABAC/CAVLC selection information embedded in the header 
information of the input bitstream, one of CABAC and CAVLC is applied. 
[00241 Fig. 14 shows the CABAC system in the reverse decoding unit 122. In Fig. 14, a 
CABAC decoding unit 161 applies binary symbol arithmetic decoding to a bitstream input 
from the storage buffer 121, and outputs the resultant as a string of binary symbols. This 
string of binary symbols is input into an inverse binarization unit 163, and the inverse 
binarization unit 163 converts the string of binary symbols into multi-valued symbols under a 
predetermined rule. The multi-valued symbols to be output from the inverse binarization unit 
163 are output from the inverse binarization unit 163 to the dequantization unit 123 and the 
motion prediction/compensation unit 128 as mode information, motion vector, and quantized 
coefficient information. A Context operation unit 1 62 calculates Context based on the string 
of binary symbols input into the inverse binarization unit 1 63 and the multi-valued symbol 
output from the inverse binarization unit 163, and inputs the Context into the CABAC 
decoding unit 161. A Context memory group 165 of the Context operation unit 162 stores 
Context which is updated, as occasion arises, during a decoding process, and the initial state 
of Context to be used for a reset. 

[0025] Next, the CAVLC system in the reverse decoding unit 122 is shown in Fig. 15. As 
shown in Fig. 15, an input bitstream from the storage buffer 121 is input into a CAVLC 
decoding unit 170. Like variable-length decoding adopted by the conventional MPEG, the 
CAVLC decoding unit 170 adopts a variable-length decoding table for the input bitstream 
and outputs mode information, motion information, and quantized transform coefficient 
information. These output information are input into the dequantization unit 123 and the 
motion prediction/compensation unit 128. A Context storage unit 171 stores information 
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already decoded in the CAVLC decoding unit 170, for example, the number of coefficients of 
non-zero in blocks already processed as well as blocks being processed, the value of a 
coefficient decoded just before this time, and so on. The CAVLC decoding unit 170 ,s able 
to change the variable-length decoding table to be applied for symbols, based on informal™ 
from the Context storage unit 171. It should be noted that the Context storage unit 171 stores 
the initial state of Context to be used for a reset as well. 

,0026] For specific operations of the CABAC shown in Fig. 1 1 and Fig. 14, an explanation 
about the CABAC is written in Final Committee Draft ISO/IEO 14496-10:2002 (section 9.2), 
the entire contents of which are hereby incorporated herein by reference. 

SUMMARY OF THE INVENTION 
10027] The applicants of the present invention have recognized certain drawbacks in the 
existing encoding and decoding systems, and improvements that can be made therein, as now 
discussed. 

[0028] When 1 picture is encoded in a video-information encoding apparatus 100, even any 
of 1 picture, slice, macroblock and block is considered as a unit of encoding, the number of 
symbols, included in the unit of encoding, to be entered into the binarization unit 131 of Fig. 
1 1 is not fixed since it depends on a video signal to be entered and encoding' conditions. 
[0029] In addition, the length of a binary data string to be output for one symbol entered into 
the binarization unit 131 is unfixed as described in section 9.2.1 of JVT FCD. For example, 
as is clear from Table 9-20 in section 9.2.1.5 of JVT FCD, the length of a binary data strxng 
for mbjypel Symbol in I slice is 1 at a minimum (at the time of Intra_4x4), and is 6 at a 
maximum. Therefore, the length of binary data output from the binarization unit 131 in 
response to one Symbol is also unfixed. 

[0030] For the above reasons, the number of pieces of binary data output from the 
binarization unit 131 in response to a symbol, included in a unit of encoding, of an input 
video signal is unfixed, and therefore a large amount of binary data may be possibly output 
from the binarization unit 131, due to the input data and encoding conditions. 
[00311 The binary data output from the binarization unit 131 is input into the CABAC 
encoding unit 133. However, since the CABAC encoding unit 133 actually needs a 
processing time longer than one clock to process one piece of input binary data, if a great 
number of pieces of binary data are input into the CABAC encoding unit 133, a large 
processing time is accordingly required. In addition, as described above, as the number of 
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pieces of binary data input into the CABAC encoding unit 133 is unfixed, the longest time for 
processing cannot be estimated. 

[0032] Therefore, in a case in which the video-information encoding apparatus 100 should 
have a guarantee of real-time processing and a fixed processing speed, it can not have the 
guarantee if a great number of pieces of binary data is input into the CABAC encoding umt 
133 or the number of pieces of binary data is unfixed. 

[0033] In addition, a bit length output from the CABAC encoding unit 1 33 in response to a 
binary data string output from the binarization unit 13 1 in response to one symbol is unfixed. 
This is because the CABAC controls the output bit length according to the occurrence 
probability of input binary data. As a result, one piece of binary data input into the CAB AC 
encoding unit 133 may be bitstream data of one bit or lower, or bitstream data of several bits, 
depending on its occurrence probability. 

[0034] Since the CABAC encoding unit 133 actually needs a processing time longer than 
one clock cycle to process one piece of bit data to be output, if a large number of pieces of bit 
data are output from the CABAC encoding unit 133, a long processing time is required and, 
accordingly as a result a mounted encoding unit needs a large processing time. In addition, 
as stated above, since the number of pieces of bit data output from the CABAC encoding unit 
133 is unfixed, it is difficult to estimate the longest time for processing. 
[0035] Therefore, in a case in which the video-information encoding apparatus 100 should 
have a guarantee of real-time processing and a fixed processing time, it can not have the 
guarantee if a large number of pieces of bit data are output from the CABAC encoding unit 
133 or the number of pieces of bit data is unfixed. 

[00361 The matter in that the number of pieces of binary data or bit data to be input to/output 
from the CABAC encoding unit 133 is unfixed in a unit of encoding, such as 1 picture, slice 
of a picture, macroblock, or a block, and may became large prevents the guarantee of a fixed 
processing time in the unit of encoding. 

[00371 Further, when 1 picture is decoded in the video-information decoding apparatus 120, 
even any of 1 picture, slice, macroblock, and block is considered as a unit of decoding, the 
number of bits of a bitstream, included in the unit of encoding, entered into the CABAC 
decoding unit 161 is unfixed because it depends on an input bitstream. 
[0038] Since the CABAC decoding unit 161 requires a processing time longer than one 
clock cycle to process one piece of input bit data, if a great number of pieces of bit data are 
input into the CABAC decoding unit 161, a large processing time is accordingly required. In 



addition, as stated above, since the number of pieces of bit data input into the CABAC 
decoding unit 161 is unfixed, the slowest processing speed can not be estimated. 
[0039] Therefore, in a case in which the video-information decoding apparatus 120 should 
have a guarantee of real-time processing and a fixed processing time, it can not have the 
guarantee if a large number of pieces of bit data are input into the CABAC decoding 
apparatus 161 or the number of pieces of bit data is unfixed. Especially, as compared with 
the video-information encoding apparatus 100, in the video-information decoding apparatus 
120, the real-time decoding and display of video information are highly demanded. 
Therefore, the fact that real-time processing can not be guaranteed is a problem. 
[0040] Accordingly, one object of the present invention is to address the above-problem. In 
the present invention, considering the above problems, the amount of data to be input 
to/output from a CABAC encoding unit and decoding unit is limited in a unit of encoding 
such as 1 picture, slice, macroblock or block, and a mechanism for the limitation is applied to 
a video-information encoding apparatus and decoding apparatus, and its bitstreams. 
[0041] Further, in the present invention, considering the above problems, uncompressed 
video data is encoded and a mechanism for the encoding is applied to a video-information 
encoding apparatus and decoding apparatus, and its bitstreams. 

[0042] Still further, in the present invention, considering the above problems, the same 
method is applied to CAVLC as well as CABAC. 

[0043] By limiting the amount of data to be input to/output from the aforementioned 
CABAC encoding unit and decoding unit and encoding uncompressed data, a video- 
information encoding apparatus and decoding apparatus can have a guarantee of a fixed 
processing time, and an apparatus with a guarantee of the processing time can be realized. In 
addition, similar effects can be obtained in CAVLC as well as CABAC. 

BRIEF DESCRIPTION OF DRAWINGS 
[0044] A more complete appreciation of the present invention and many of the attendant 
advantages thereof will be readily obtained as the same becomes better understood by 
reference to the following detailed description when considered in connection with the 
accompanying drawings, wherein: 

[0045] Fig. 1 shows an example of a construction of a video-information encoding apparatus 
of the present invention; 
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[0046] Fig. 2 shows an example of a construction of a video-information encoding apparatus 
of the present invention; 

[0047] Fig. 3 shows an example of a construetion of a video-information encoding apparatus 
of the present invention; 

[0048] Fig. 4 shows an example of a construetion of a video-information encoding apparatus 
of the present invention; 

[0049] Fig. 5 shows an example of a construction of a video-information encoding apparatus 
of the present invention; 

[0050] Fig. 6 shows an example of a construction of a video-information decoding apparatus 
of the present invention; 

[0051] Fig. 7 shows an example of a construction of macroblock processing unit of the 
present invention; 

[00521 Fig. 8 shows an example of a construction of abackground video-information 
encoding apparatus; 

[00531 Fig. 9 shows an example of a construction of a background video-information 
decoding apparatus; 

[00541 Fig. 1 0 shows an example of a construction of a variable-length coding unit m 
background JVT; 

[00551 Fig. 11 shows an example of a construction of a CABAC encoding umt in 
background JVT; 

[00561 Fig. 12 shows an example of a construction of a CAVLC encoding unit in 
background JVT; 

[00571 Fig. 1 3 shows an example of a construction of a variable-length decoding umt in 
background JVT; 

[0058] Fig. 14 shows an example of a construction of a CABAC decoding unit in 
background JVT; and 

[0059] Fig. 15 shows an example of a construction of a CAVLC decoding unit in 
background JVT. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0060] Embodiments of the present invention will be explained with reference to the 
attached drawings hereinafter, in which like reference numerals indicate identical or similar 
elements throughout. 
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[00611 Fig. 1 shows an embodiment of a video encoding apparatus 10 in the present 
invention. In the apparatus 10 of Fig. 1, video signa.s to be encoded are input, and an 
encoded bitstream is output. The apparatus 10 is composed of an input buffer 11, a transform 
processing section 12, a CABAC processing section 13, a limitation control unit 14, and an 
output buffer 1 5. The input buffer 1 1 partitions input video into macroblocks and outputs a 
macroblock, and when the processing of the maeroblock is finished a. the latter stage, outputs 
a next macroblock. The transform processing section 12 processes an input maeroblock 
image and outputs header information and quantized coefficient information to the CABAC 
processing seetion 1 3. Specifically, a parameter setting unit 16 sets header information such 
as mode information and motion vector information of a macroblock, and a quantization 
parameter, and outputs the value (symbol) to a prediction unit 17, a DCT unit 1 8, a 
quantization unit 19, and the CABAC processing section 13. The parameter setting unrt 16 
can set and output header information of a slice and picture, as well as header informatton of 
a macroblock, and these information are referred to as header information altogether. Also, a 
motion compensation in the prediction unit 17, a DCT transform in the DCT unit 18, and a 
quantization process in the quantization unit 19 are applied to input signals from their former 
stages by reference to an input signal fiom the parameter setting unit 16. 
[0062] In the CABAC processing section 13, header information and quantized coefficient 
information are input as symbol data, subjected to arithmetic coding, and output as bit data. 
Specifically, the input symbol data is transformed into a binary data string by a binarization 
unit 20 and the binary data is entropy-coded by a CABAC encoding unit 22 based on 
Context information from a Context operation unit 21 . The Context operation unit 21 
updates Context based on the symbol data input into the binarization unit 20 and the binary 
data output from the binarization unit 20, and also outputs the Context information to the 
CABAC encoding unit 22. 

[0063] The limitation control unit 14 has a first counter for the number of pieces of binary 
data input into the CABAC encoding unit 22 and a second counter for the number of pieces 
of output bit data (bit counter 25). The limitation control unit 14 increases the first counter 
by one every time a binary data is input into the CABAC encoding unit 22, and increases the 
second counter by one every time a bit data is output from the CABAC encoding unit 22. 
These counters reset to 0 every time the processing of the beginning of the macroblock starts. 
Thereby, the numbers of pieces of input data and output data in/from the CABAC encoding 
unit 22 can be counted. 
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,0064) When one or both of these eounters shows a nnmber exeeeding a preset threshold 
valne the limitation eontrol unit 14 outputs a signal (hereinafter referred to as a "reencoding 
signal") indieating that the data to be encoded is invalid, to the output buffer 15, the Context 
operation unit 21, and the parameter setting unit 16. The parameter setting unit 16whieh 
received the reencoding signal resets an encoding parameter so as no, to exceed the threshold 
value and performs a reencoding process on macroblock dam to be encoded. In addmon, the 
Context operation unit 21 has a Context memory group 23, and this Context memory group 
23 stores Context which is updated, as required, during an encoding process and the initial 
state of Context which is used for a reset, as in the Context memory group 135 shown m F.g. 
„ as a background art, and also can store the state of Context just before the dam processing 
of a macroblock. Therefore, the Context operation unit 21 which received a reencodmg 
signal rewrites the state of Context being stored, to the value of Context stored in the memory 
newly added, thereby making it possible to restore the state of Context jus. before the update 
performed based on macroblock data to be encoded. In addition, the output buffer 15 which 
received the reencoding signal deletes all of bit data of a macroblock to be encoded and 
stored internally, and waits for an input of macroblock data encoded with a new encodmg 
parameter. In addition, if any counter of the limitation control unit 14 does not show a value 
exceeding the preset threshold value when the encoding process of a target macroblock 
finishes, the bit data of the target macroblock in the output buffer 15 can be output as a 
bitstream. 

[00651 In the apparatus 10 of Fig. 1 described above, reset of the counters of the limitation 
control unit 14 a. the beginning of a macroblock means that the number of pieces of binary 
data to be input to the CABAC encoding unit 22 and the number of pieces of b,t data to be 
output are monitored and limited on a macrobloek basis, and by setting timing for this reset 
on a block basis in a macroblock, the number of pieces of data can be monitored and hunted 
on a block basis. Similarly, by performing a reset on a slice basis, the number of pieces of 
' dan, can be monitored and limited on a slice basis, and by performing a reset on a p.cture 
basis the number of pieces of data can be monitored and limited on a picture basis. When a 
unit for managing and limiting tire number of pieces of data is changed, the Context memory 
group 23 of the Context operation unit 21 stores* Context value in a unit of encodmg just 
before the change at the same time, and therefore the state in die unit of encoding jus. before 
the change is restored as the state of Context. In addition, bit da.aof .he output buffer 15 is 
also deleted every unit of encoding. 
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[0066] Also, restoration to a predetermined initial value, which is stored in the Context 
memory group 23, can be also performed, not to the Context value in the unit of encoding 
just before the change, which is stored in the Context memory group 23. 
[0067] In the apparatus 10 of Fig. 1 described so far, although the limitation control unit 14 
has two counters, threshold values set in these counters can be independently set to various 
values only a data count of one of these two counters can be monitored and a data count of 
the other can be ignored, or the limitation control unit 14 can be constructed without 
counters. 

[0068] in this apparatus 10, the maximum amount of data to be input into/output from the 
CABAC encoding unit can be limited for one time macroblock processing, which allows the 
requested time for one time processing of macroblock to be used satisfactorily. In addmon, a 
bitstream to be decoded within a requested processing time can be output. 
[0069] Now Fig. 7 shows a unit, a macroblock processing unit, representing the transform 
processing section 12 and the CABAC processing section 13 of Fig. 1, for the following 
description. The macroblock processing unit described below performs the same process as a 
unit in which the transform processing unit 12 and the CABAC processing unit 13 of F,g. 1 
are connected in parallel. 

[0070] In the apparatus 10 shown in Fig. 1, the transform processing section 12 has to set a 
new encoding parameter and encode a target macroblock every time the limitation control 
unit 14 outputs a reencoding signal, and further the counters of the limitation control unit 14 
may exceed a threshold value repeatedly, due to data obtained by a re-set parameter. In this 
case, the encoding processes are successively performed on one macroblock several times, 
which takes a long time to encode 1 picture. 

[0071] As another embodiment of a video encoding apparatus in the present invention, an 
example in which a target macroblock is encoded in parallel with different parameters is 
shown in Fig. 2 as a second embodiment. 

[0072] In an apparatus 30 of Fig. 2, similarly to the apparatus 10 of Fig. 1, video signals to 
be encoded are input and an encoded bitstream is output. The apparatus 30 of Fig. 2 is 
composed of an input buffer 31, macroblock processing units 32-1 to 32-N capable of 
performing parallel encoding processes at N stages with N different encoding parameters, 
corresponding output buffers 33-1 to 33-N, a limitation control/route selection unit 34, and a 
switcher 35. 
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[0073] In the apparatus 30 of Fig. 2, N different encoding parameters are set for a 
macroblock to be encoded, and the encoding processes with these encoding parameters are 
performed in parallel by the macroblock processing units 32-1 to 32-N, and their outputs are 
stored in the output buffers 33-1 to 33-N. 

[0074] The limitation control/route selection unit 34 has two input/output data counters (bit 
counter 36) for the CABAC encoding unit corresponding to the macroblock processing units 
32-1 to 32-N, and selects an encoding route which does not make its counter exceed a 
threshold value and therefore is the most efficient encoding route, out of the N parallel routes, 
and selects a line for output from the switcher 35. 

[0075] The detailed operation of the apparatus 30 of Fig. 2 and variations of a unit of 
encoding are the same as the apparatus 10 of Fig. 1. 

[0076] In this apparatus 30, the maximum amount of data to be input into/output from the 
CABAC encoder at the time of encoding can be limited, so that the requested encoding 
processing time can be used satisfactorily. In addition, a bitstream that can be decoded 
within a requested processing time can be output. 

[0077] Next, Fig. 3 shows another embodiment of a video encoding apparatus of the present 
invention. In this embodiment, in addition to the embodiment of Fig. 1, a route for 
uncompressed encoded data, that is for encoding uncompressed raw data of an input 
macroblock, is provided. 

[0078] What is different between the apparatus 40 of Fig. 3 and the apparatus 10 of Fig. 1 is 
that macroblock video data is input not only into a macroblock processing unit 41 but also 
into an uncompression encoding unit 43. The uncompression encoding unit 43 outputs data 
of input video information which is not subjected to a transform process and entropy coding, 
that is raw data to the output buffer B 44. A limitation control/route selection unit 45 
manages the amount of data to be input to/output from the CABAC encoder by using the bit 
counter 49, and if the data being monitored exceeds a threshold value, the switcher 46 selects 
an input from the output buffer B 44 for output, similar to the limitation control unit 14 of 
Fig. 1. If the data does not exceed a threshold value, an output from either output buffer A 42 
or output buffer B 44 can be selected. 

[0079] In a case in which the limitation control/route selection unit 45 selects the output 
buffer B 44, that is raw data, the Context operation unit of the macroblock processing unit 41 
is informed of this matter, and the state just before the processing of the macroblock which 
was processed as raw data is restored for the Context value of the Context operation unit, by 
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using the Context value just before the macroblock was processed, which is stored in the 
Context memory group. 

[0080] Also, in restoration of Context when the macroblock is processed as raw data, a 
predetermined initial state can be restored as well. 

[0081] Data indicating whether a macroblock has been encoded as raw data or not is 
embedded into the header information of a bitstream to be output. 

[00821 In a case in which raw data has been encoded, the CABAC encoding unit performs 
the terminal process before outputting the raw data as a bitstream. 
[00831 In addition, the uncompression encoding unit 43 is not limited only to an 
uncompression encoding unit that outputs raw data, but can also be another type of 
compression unit such as a DPCM encoding unit. 

[0084] Other operations of the apparatus 40 of Fig. 3 and variations of a unit of encoding are 
the same as the apparatus 10 of Fig. 1. 

[00851 In this apparatus 40, the maximum amount of data to be input into/output from the 
CABAC encoding unit during encoding can be limited, so that a requested encoding 
processing time can be used satisfactorily. In addition, a bitstream that can be decoded 
within a requested processing time can be output. 

[0086] Next, Fig. 4 shows an apparatus 50 as another embodiment of a video encoding 
apparatus of the present invention. In this embodiment, in addition to the apparatus 30 of 
Fig. 2, a route for data to be uncompressed and encoded, that is for uncompressing and 
encoding raw data of input macroblock, is provided. 

[00871 Since operations of the common units of the apparatus of Fig. 4 with the apparatus of 
Fig. 2 are almost the same as the apparatus of Fig. 1, only different operations will be 
described. In this embodiment macroblock video data is input not only to macroblock 
processing units 51-1 to 51-N but also to an uncompression encoding unit 58. The 
uncompression encoding unit 58 outputs data of input video information which is not 
subjected to a transform process and entropy coding, that is raw data, to an output buffer B 
59 A limitation control/route selection unit 53 monitors a bit counter 55, as in the limitation 
control/route selection unit 34 of Fig. 2, and if a bit counter for all routes 1 -N (any of two) 
exceeds a preset threshold value, a signal selection unit 54 selects and outputs an input from 
the output buffer B 59. And if the bit counter 55 does not exceed the threshold value, the 
signal selection unit 54 can select any of outputs from the output buffers A 52-1 - A 52-N 
and the output buffer B 59. 
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[00881 Also, if the signal selection unit 54 selects raw data from the output buffer B 59, the 
state of Context just before a macroblock was processed, which is stored in the Context 
memory group, is restored as the state of Context of the Context operation units of the 
macroblock processing units 51-1 to 51-N. It should be noted that a predetermined initial 
value is able to be restored in this restoration, as described for the apparatus 10 of Fig. 1. 
[0089] When the signal selection unit 54 selects an output buffer A 52-1 out of the output 
buffers A 52-1 to A 52-N, not raw data from the output buffer, on the contrary, the state of 
Context of the Context operation unit of the macroblock processing unit 51-1 is copied to the 
Context operation units of the other macroblock processing units 51-1 to 51-N. This is 
because all the states of the Context of the Context operation units should be the same for the 
time when encoding of a next macroblock starts. It should be noted that the uncompression 
encoding unit 58 is not limited to only an uncompression processing unit that outputs raw 
data, but can also be another type of compression unit such as a DPCM encoding unit. 
[0090] Other detailed operations of the device 50 of Fig. 4 and variations of a unit of 
encoding are the same as the apparatus 1 0 of Fig. 1 . 

[0091] In this apparatus 50, the maximum amount of data to be input to/output from the 
CABAC encoding unit can be limited during encoding, so that a requested encoding 
processing time can be used satisfactorily. In addition, a bitstream which can be encoded 
within a requested processing time can be output. 

[0092] Next, Fig. 5 shows an apparatus 60 using CAVLC, not CABAC, as the reverse 
encoding unit 106 of Fig. 8. This apparatus 60 has a CAVLC processing unit 63, instead of 
the CABAC processing unit 13 of the apparatus 10 of Fig. 1, and performs the same 
operation, except for the CAVLC processing unit 63 and the limitation control unit 64. 
Therefore, only the operation of the CAVLC processing unit 63 and the limitation control 
unit 64 are described in this section. 

[0093] In the CAVLC processing unit 63, header information and quantized coefficient 
information input as symbol data are variable-length encoded with a variable length table, 
similar to a conventional MPEG2, and are output as bit data. The CAVLC processing unit 63 
is composed of the CAVLC encoding unit and the Context storage unit described for the 
background art in Fig. 12, and the CAVLC processing unit 63 of the present invention can 
store the state of Context just before a macroblock is encoded to restore the state just before 
the macroblock is encoded when a reencoding signal arrivals, in addition to storing 
information encoded by the CAVLC encoding unit, for example, the number of non-zero 
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coefficients in each block of not only blocks being processed but also blocks already 
processed and a valne of a coefficient encoded just before this time, as in the background 
storage unit. The CAVLC encoding unit can change a variable-length coding table to be 
applied to a symbol, on the basis of information from this Context storage unit. Note that, the 
Context storage unit stores the initial state of Context which is used for a reset. 
10094) The limitation control unit 64 has one counter (bit counter 75) for the number of 
pieces of bit data output from the CAVLC processing unit 63, and increases this counter by 
one every time the CAVLC processing unit 6 outputs bit data. This counter resets to zero 
when the processing of beginning of a macroblock starts. Therefore, the number of pieces of 
output data from the CAVLC processing unit 63 can be counted for each macroblock. 
.0095] If this counter 75 exceeds a predetermined threshold value, the limitation control untt 
64 outputs a signal (referred to as a "reencoding signal" hereinafter), indicating that the data 
to be encoded is invalid, to an output buffer 65 and a parameter setting unit 66. The 
parameter setting unit 66 that receives this reencoding signal re-sets an encodmg parameter 
not to exceed the threshold value and performs reencoding on macroblock data to be 
encoded. In addition, the output buffer 65 that received the reencoding signal deletes all btt 
data of the macroblock to be encoded, being stored therein, and waits for macroblock data 
encoded with a new encoding parameter to be input. 

[0096] Other detailed operations of the apparatus 60 of Fig. 5 and variations of a untt of 
encoding are the same as the apparatus lOofFig. 1. 

[00971 In this apparatus 60, the maximum amount of data output from the CAV AL encodmg 
u„i, can be limited for one-time macroblock processing, so that a requested macroblock 
processing time can be used satisfactorily. In addition, a bitstream which can be decoded 
within a requested processing time can be output. 

[0098] Further, in not only the apparatus of Fig. 1 but also the apparatuses of Fig. 2 to F,g. 4, 
the CAVLC processing unit can be used instead of the CABAC processing unit and rts 
operation is the same of that described now in this embodiment. However, if a macroblock ts 
encoded as raw data, the CAVLC processing unit does not have Context for the macroblock, 
so that a method for updating Context when raw data is encoded should be defined. Vanous 
kinds of methods can be applied, provided that an encoding apparatus and a decodmg 
apparatus are synchronized with each other. For example, the number of non-zero 
coefficients existing in blocks of a macroblock encoded as raw data can be taken to 1 5. In 
this apparatus, the maximum amount of data to be output from the CAVLC encoding untt can 
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be limited during eneoding, so that a requested encoding processing time can be used 
satisfactorily. In addition, a bitstream which can be decoded within a requested processing 
time can be output. 

[0099] Next, Fig. 6 shows an apparatus 80 of a video-information decoding apparatus of the 
present invention corresponding to the apparatuses of Fig. 1 to Fig. 4. It should be noted that 
the apparatuses of Fig. 1 and Fig. 2 do not have an uncompression encoding umt and route 
for the unit, so that the apparatus 80 of Fig. 6 does not select a route to an uncompression 
decoding unit 88. In a ease in which such a situation is clear, the uncompression decodmg 
unit 88 and the route for the unit may not be provided. 

[0100] In the apparatus 80 of Fig. 6, a bitstream to be decoded is input and a decoded video 
signal is output. The apparatus 80 of Fig. 6 is composed of route selection units 81, 85, an 
eneoding method judgement unit 84, an inverse transform processing unit 83, a CABAC 
processing unit 82, a limitation control unit 86, and the uncompression decoding umt 88. 
[0101] First, when processing of each macroblock starts, the route selection units 81, 85 
select a route of the CABAC processing unit 82. The CABAC processing unit 82 decodes a 
symbol embedded into a bitstream, indicating whether a macroblock is raw data or not, 
before decoding the macroblock from the input bitstream, and if the encoding method 
judgement unit 84 judges that i, is raw data, the route selection units 81, 85 select a route of 
the uncompression decoding unit 88, to output the output from the uncompresston decodmg 
unit 88 as a video signal. At this time, the uncompression decoding unit 88 obtains v.deo 
data through fixed-length decoding. If the uncompression encoding unit 88 is selected, the 
state of Context of the Context operation unit 92 of the CABAC processing unit 82 may not 
be changed, may be initialized with a predetermined value, or may be changed with another 
rule provided that it is synchronized with the operation of the CABAC processing unit on an 
encoding apparatus side. In addition, at this time, a predictor used for decoding a macroblock 
in the same picture to be decoded later is set to a predetermined value. For example, the 
motion vector of a macroblock uncompression-decoded is set to 0, and a macroblock type ,. 
set to an intra-encoding. This predictor value may have any value, provided that it >s 
synthesized with an encoding apparatus side. 

[0102] If the encoding method judgement unit 84 selects that macroblock data is processed 
by the CABAC processing unit 82, an input bitstream is input into the CABAC processmg 
unit 82 successively. 
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[0103] The CABAC processing unit 82 decodes and ontputs header information and 
quantized coefficient information from an input bitstream, as symbol data. Specifically, the 
input bitstream is entropy-decoded by the CABAC decoding unit 90 based on Context 
information from the Context operation unit 92, and the output binary symbol stnng .s 
transformed into symbol data by the inverse binarization unit 91 . The Context operation umt 
92 updates Context based on the binary data input into the inverse binarization umt 91 and 
symbol data output ftom the inverse binarization unit 91 , and outputs the Context informahon 
,0 the CABAC decoding unit 90. The operation of me CABAC processing unit 88 is under 
description of section 9.2 of JVT FCD noted in the background art. 

[0104] The inverse transform processing unit 83 performs dequantization, inverse DCT, and 
motion compensation on input header information and quantized coefficient information, to 
decode and output video signals. 

[0105] The limitation control unit 86 has a first counter for the number of pieces of bit data 
to be input into the CABAC decoding unit 90 and a second counter for the number of pieces 
of binary data output (bit counter 13), and increases the first counter by one every time bit 
data is input into the CABAC decoding unit 90, and increases the second counter by one 
every time binary data is output from the CABAC decoding unit 90. These counters reset to 
zero when the processing of the beginning of the macroblock starts. Thereby, the number of 
pieces of input data and output data of each macroblock in/from the CABAC decoding unit 
90 can be counted. 

[0106] If any of these counters exceeds a preset threshold value, the limitation control unit 
86 performs error processing. This error processing can stop a decoding process once and 
start the decoding process again after a slice header or picture header arrives, or can only 
make a warning and keep the decoding process going. In addition, the decoding process can 
be kept without the error processing. 

[0107] In this apparatus 80, the amount of data to be input into/output from the CABAC 
decoding unit 90 during decoding can be controlled, so that even the amount of data 
exceeding the maximum amount of data is input or output, the error processing or the like can 
be performed to use a requested decoding processing time satisfactorily. 
[0108] In addition, the limitation control unit 86 is not necessarily mounted in the apparatus 
80. In this case, the amount of data input and output is not monitored in the CABAC 
encoding unit 90. 



19 



[0109] Also, although the apparatus 80 shows an embodiment of a video information 
decoding apparatus of the present invention in a case in which CABAC is applied as entropy 
decoding, the CAVLC processing unit can be used instead of the CABAC processing unit as 
shown in the embodiment for a video encoding apparatus. The description for its actual 
processes will be omitted because the apparatus are very similar, as described in an 
embodiment for an encoding apparatus. Note that, similarly to the encoding apparatus, a 
method of updating Context in CAVLC for a case in which macroblock is encoded as raw 
data is previously defined. 

[01101 Next, an embodiment of abitstream encoded according to the present invention will 
be shown. As described so far, both data compressed in a bitstream and raw data can be 
encoded in the present invention. Therefore, header information indicating whether the 
macroblock has been encoded as raw data or not is added, which is followed by raw data or 
compressed bit data. The information indicating whether a macroblock has been encoded as 
raw data or not is different depending on a macroblock type that is one of macroblock header 
information. Conversely, a bitstream in the present invention can include macroblocks 
processed in different encoding methods. 

[0111] Further, this specification shows a case in which information specifying the encoding 
method applied to a macroblock is added as header information of the macroblock. If this 
specification information is added in a slice header or a picture header, encoding methods can 
be mixed and the encoding methods can be specified on a slice or picture basis. 
[0112] If header information (for example, macroblock type) is encoded by CABAC and the 
raw data (that is, a fixed-length bit string) is encoded, a bit subjected to the terminal process 
of the CABAC is inserted in a bitstream of the present invention before the raw data is 
encoded. 

[0113] Further, in a case in which a bitstream is encoded by CABAC, the bitstream is 
composed of data that does not make any bit counter for input and output of the CABAC 
encoding unit and decoding unit exceed.a preset threshold value preset. In addition, in a case 
in which the bitstream is encoded, the bitstream is composed of data that does not make any 
bit counter for output of CAVLC encoding unit and input of decoding unit exceed a preset 
threshold value. As a result, with bitstreams of the present invention, a video-information 
encoding apparatus and a video-information decoding apparatus have a guarantee of a fixed 
decoding processing time. 
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[0114) By limiting the amount of data to be input to/output from a CABAC encoding nmt 
and decoding unit and by encoding uncompressed data, a video-information encoding 
apparatus and decoding apparatus have a guarantee of a fixed processing time, thus maktng t, 
possible to realize a device with a guarantee of the processing time. In addition, the same 
effects can be obtained in a case in which CAVLC is used instead of CABAC. 
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